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1. Problem statement 

This note gives a proof of the fact that one can recover a first-order LTI system (i.e. an ex- 
ponential impulse response) from the upper-diagnonal part of the corresponding Hankel matrix 
using the nuclear norm heuristic 13]. This elementary problem finds application in tasks of system 
identification, as e.g in Ijj. Note that this problem resorts traditionally to the theory of Kalman- 
Ho on realization, see e.g. H. However, the use of those results leads to sub-optimal estimates, 
and recently there is interest in using suitable convex relaxations to address this issue. The use of 
the nuclear norm is seen as a good candidate for a convex relaxation, and the present manuscript 
gives sound arguments why this works in a first order case. However, negative empirical results on 
the use of the nuclear norm for general LTI systems with noisy observations were reported in El. 
Recently, the use of 'atomic norms' in this setting was advocated, see ifSll . 

Theorem 1. Given h £ R and -1 < h < I, define the vector h G M", h = [1, h,h'^,..., /i""!]^, 
matrices G,Go,G G M"^", and Gq = hh^. Let T-Ln be the set of all n x n Hankel matrices, such 
that G, Gq G Ti. The optimization problem is as follow: 

(1) G = argmin ||G||* s.t. G{i,j) = Go{i,j),yi + j<n + l, 

G€Hn 

where \\ ■ j|* represents the nuclear norm of a matrix. Then we have that 

G = Gq. 

The remainder of this note gives a proof of this statement. 

2. Proof of Theorem 1 

Lemma 1 gives a sufficient condition for recovery of Gq to take place by solving (O. Lemma 2 
and Lemma 3 will then assert that this condition is satisfied under the assumptions of Theorem 1 . 
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In the following, define hi = ||^|^- Let Hn € 7^"^" be a Hankel matrix so that the elements in 
the upper triangle part and the main anti-diagonal are all zero. For instance, when n = 3, 

Hr, = 




for vi,V2 S K. Specifically, we define H for any Go as in Theorem 1 , and for any G € Tin which 
satisfies the equality constraints of (d): 

(2) H = Go-G, 

Indeed, this H has an lower anti-diagonal form by construction. We need the following property 
of the nuclear norm 

Proposition 1. Given any A, M £ W^^"- matrix, then 

(3) Pll* = sup ir{MA). 

\\M\\2<1 

Lemma 1. Define P,Q e M"^" as P = (hihf ) and Q = In - P, and let H as in eq. (|2]). If 

(4) \tT{PH)\<\\QHQ\l, 
then the optimization problem (|7|) recovers Gq exactly.. 

Proof. Let V G ^"^("^-1) be a basis for Q. Hence the sub-gradients of || • ||^, at (hh^) is equal to 
the following set ( see e.g. f3|): 

(5) S = {S = (hihf ) + VBV^, yP : \\B\\2 < 1} . 

So, we need to prove that for any H which satisfies the conditions as mentioned in the theorem, 
there exists one element in S, i.e. one B with |1-B||2 < 1, such that 

{H, (hihf ) + VBV^) > 

^ triHP) > {H, -VBV^) 

(6) ^ triHP) > {V^HV, -B). 
Now, from assumption dUl, we have that 

(7) ltr(i7P)| < IIQi^Q^II,, 

and since by definition of Q as a projection matrix onto an n — 1 dimensional subspace, one has 
that 

(8) WQHQ'^W, = \\V^HV\\,, 
it follows that 

(9) \ii{HP)\<\\V^HV\U. 

Furthermore, it follows from Proposition 1 that there exists a matrix Bi with II2 < 1, such that 

(10) \\V'^HV\U = {V^HV.Bi) , 
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or 

(11) \tj:{HP)\ < {V^HV,Bi) . 
Conversely, one gets 

(12) tr(FP) > - {V^HV, Bi) = {V^HV, -Bi), 

and hence, the inequality ^ holds for Bi, which proves the result. □ 

Remark 1. In the Lemma, the statement is given in terms of the Q matrix (rather than V) since it 
will be more convenient to work with Q in the forthcoming proofs. 

Lemma 2. Define P,Q e M"^" as P = (hihf ) and Q = In - P, and let H as in eq. (|2]). Then 

(13) |tr(i/P)| < IIQFQII,. 
Proof. We distinguish between two cases, either 

(14) ii{HP) < \\QHQ\\„ 
or 

-ti{HP) < WQHQW,. 

We will give a derivation of eq. ([T4l ). the latter equation follows along the same lines. Decom- 
pose H as a linear combination of basis matrices {G, } as follows. For dimension n, H can be 
represented as 

n-l 



(15) H = Y,ViGi 



where {Gi} are the basis matrices with the elements of the ith lower anti-diagonal equal to 1 and 
the others equal to zero. For example for n = 3, one has 



(16) H = \ vi \ =vi \ 1 \ +V2 






for vi,V2 £ M. Define the vector v = [vi,V2, ■ ■ ■ , Vn-i]'^ G ^. From eq. ((Sjl, it follows that eq. 
([Mil is equivalent to prove the existence of M E M"^", for any v such that 

n-l 

(17) sup y^VitviQGiQM -GiP)>0. 

\\Mh<l 

Since both sides of eq. (fT4l ) scale linearly, one can - without loss of generality - consider the case 
that II 

^ 1 1 oo — 1 ■ Then, we need to prove the following inequality 

n-l 

(18) inf sup y^VitviQGiQM -GiP)>0. 

Il'"l|txj<l ||M||2<1 
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Application of the minimax theory as in [2] gives us that 



inf sup 

lbll^<l ||A/||2< 



> sup inf trlQiyv^GAQM-iyviGAp]. 



So, if we can prove that 
(19) 



n-l 



sup inf V Vi tiiQGiQM - dP) > 0, 

||A-/||2<lll''ll°o<l~^ 

then we are done. Now, define for a given matrix M 



n-l 



Jn{M)= inf Y ViiiiQGiQM -GiP). 

If lloo<l 



1=1 

The infimum is obtained by setting the different elements t;^ for i = 1, ... ,n — 1 as follows. 
If tr{QGiQM — GiP) > 0, then the minimal value of Jn{M) is obtained when Vi equals — 1. 
Conversely, if tx{QGiQM — GiP) < 0, then the minimal value of Jn{M) is obtained when 
Vi equals 1. If tv{QGiQM — GiP) = 0, then Vi can be chosen arbitrary. Adding this for all 
z = 1 , . . . , n — 1 implies that 

sup Jn{M) < 0. 

I|A^I|2<1 

So, we are left to prove that supp/||2<i Jn{M) = 0. In order to prove this, we need to prove the 
existence of a M with \\M\\2 < 1 such that 



tiiQGiQM - GiP) = 0, Vi = 1, 2, . . . , n - 1. 
This follows from the following Lemma. 

Lemma 3. Given —l<h<l. There exists a matrix M G R"^" with \\M\\2 < 1, such that 

(20) ix{QGiQM - GiP) = 0, Vi = 1, 2, . . . , n - 1, 

where the matrices Gi,H,P,Q G M"^" are defined as before. 

Proof. We will give a construction of such a matrix M. Let i? > be defined as 



□ 



We construct two matrices Qi G 



and Q2 G 



which satisfy the following relations: 



R^iQi + Q2) = R^Q = RHin -P) = R^ln 



1 


h 


■ 




h 




■ 








■ 




n-l 
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where I„ is the identity matrix of size n, and 
R\Qi - Q2) 

-h" -h"+^ ■■■ -/i^""^ h-\ h?i^" 

+ i -h" + '- ; -h^"-^ h + ■ ■ ■ + h""^-' -1 

: -h^"*^ /iH h^^""^ -1 -h 

: -/i^''-^ h H h -1 ; ; 

/i + — h/i^""^ -1 : : -/i"-^ 

/iH hft^""-'* -1 -/i ■•■ -;t""=' -/t""^ 

These matrices Qi, (52 obey the following properties: 

• Fact 1: 

{Qi + Q2){Qi - Q2) = {Qi - Q2){Qi + Q2). 

• Fact 2: 

(21) (Qi + Q2)(Qi + Q2) = (Qi - Q2)iQi - Q2). 



Fact 3: 
Fact 4: 



Q1Q2 = Q2Qi =0. 



Q\ = Q17 Q2 = Q2- 

Basically, Fact 3 says that matrices Qi and Q2 are orthogonal to each other, and Fact 4 says 
that Qi and Q2 are all projection matrices. As Qi and Q2 are orthogonal to each other and are all 
projection matrices, hence the matrix (Qi — Q2) will have eigenvalues either 1,-1 or 0. This means 
in turn that the spectral norm of {Qi — Q2) is lower than 1. This leads us to consider the following 
choice of M: 

(22) M = - Q2). 

Now we will prove that this matrix M satisfies all the conditions given in the Lemma based on 
these Facts. First, notice that the equalities of (l20l ) are equivalent to 

(23) tr {GiiQMQ - P)) = 0, Vi = 1, 2, . . . , n - 1. 
The terms in eq. (1231 ) can be written as follows: 

QMQ -P = -h^iQi + Q2){Qi - Q2){Qi + Q2)-P 
= -h^{Qi-Q2)-P 
= M- P. 



Notice that M will have the same elements as P in the lower anti-diagonal part, so eq. (1231 ) holds. 
Since also ||M||2 = < 1, also eq. (l20l ) holds as desired for this choice of M. 



We are left to prove the facts 1,2,3 and 4. Facts 1, 3 and 4 are given by the following reasoning. 
For Fact 1, we need to notice that the vector h (which is defined in Theorem 1) lies in the null space 
of the matrix Qi — Q2. For Fact 3, notice that it is a direct result of Fact 1 and Fact 2 if we expand 
the terms. For Fact 4, we need to notice that the eigenvalues of Qi and Q2 equal either or 1. 



ON THE NUCLEAR NORM HEURISTIC FOR A HANKEL MATRIX RECOVERY PROBLEM 



Fact 2 requires further work: the equality of (1211) follows from 

R\Qi + Q2)RHQi + Q2) = R\Qi - Q2)RHQi - Q2). 

By construction, we get that 

R\Qi + Q2)R\Qi + Q2) = R^In - R\hh^). 

We will give expression of the elements of the matrix R^{Qi — Q2)R^{Qi — Q2) by calculating 
the off-diagonal entries and the on-diagonal entries separately. 

• Ojf -diagonal elements (except for the last column and last row). Take for any 1 < A;2; < 
n the corresponding columns from the matrix A = R^{Qi — Q2) as 



A,, 



-h 



n+fci 



_j^2n—3+ki—k2 _j^2n—2+ki—k2 

_^2n— l+fci— fc2 _j^2n+ki—k2 



n-1 



-h 



2n-2 



and 



A 



k2 



n-1 



•n+k2 



_^2n-2 



i=l 



^-h^2-k,-2^_j^k2-k^-l^_J^k2- 



So, the {ki,k2) and {k2,ki) elements of R^{Qi — Q2)R^{Qi — Q2) are given by the 
inner-product: 

Af^^ Afc2 = {h + h^ + ■■■ + + /j2n-3)(_^2n-l-fc2+fci _ /jfca-fei-l) 

_j_^^2n-fc2+A;i _|_ ^2n-A;2+fci+2 _j_ . . . _j_ ^2n— 4+fc2-A;i 
_|_^^2n+fc2+fci _j_ ^2n+fc2+fci+2 _j_ . . . _j_ ^4n— 4— fc2+A,-i -j 
^(/jfc2-fcl ^ /jfc2-fel+2 + . . . + /jfcl+fe2-2)_ 

Reorganizing the equation, we get that 



Al A,, 



k2 



_^^2n— 2+fc2-fci _|_ i^2n+k2—ki _|_ 



_|_ j^2n—2+k2+ki ^ 
_Q^>^2+ki j^k2+ki+2 _|_ . . . _|_ ^2n-4+fe2-fci^ 
_ _|^/j^2+fei _j_ j^k2+ki+2 _j_ . . . _|_ j^2n—2+k2+ki^ 

Off-diagonal elements ( in the last column and last row ). Take for any 1 < k < n and n the 
corresponding columns from the matrix A = R^{Qi — Q2) as: 

T 



At 



-h 



n-1 

2n-2 i,2i-l 



i=l 



,-1, 



-h 
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and 



■n-l - T 



h^'-\ -1, ■ ■ ■ , -/^""''"^ • • • , 

J=l 

So, the (A;, n) and (n, fc) elements of R^{Qi — Q2)R'^{Qi — Q2) is given by their inner- 
product 





f 1 _j_ _|_ _ _ 






-fc+1 _j_ _|_ ^ 


• • + /i3"-fc-5 




fc— 1 _j_ ^n— fc+1 _j_ 


• • + /i3"-fc-5 




fc— 1 _|_ ^n— fc+1 _j_ 





Reorganizing the equation, we get that 

AlAn = + + + 

• r/je (A;, A;) entries where 1 < k < n. We need to verify the following equation holds: 

|-^2n+2fc _|_ ^2n+2fc+2 _|_ . . . _|_ /j4n-4j 

+ (/l + /l^ + . . . + /l2"-3)2 + (1 + /j2 ^ . . . ^ /j2fc-2) 

= (1 + /,2 + . . . + /,2n-2)2 _ (1 + ;^2 ^ . . . ^ /i2"-2);^2fc^ 

which is equivalent to prove that 

(24) (1 + + . . . + /i4"-4) + (/j + /j3 + . . . ^ /i2"-3)2 = (1 + /j2 ^ . . . ^ /i2«-2)2. 

This is in turn equivalent to the following equality 

(1 - h^){l - /l4"-2) + /i2(l _ /j2n-2)2 ^ _ /j2n)2 
^ (l-/l2)2 ~ (1 -/l2)2 

4^1 + /l4"-2/l2" = (l-/i2")2. 

• r/je (n, n) ewfry. We need to verify that the following equation holds: 



2n-4N 



= (1 + /l2 + . . . + ;,2n-2)2 _ (1 + /,2 ^ . . . ^ ^2n-2)^2n~2^ 

which is equivalent to prove that 

il + h^ + --- + /l") + {h + h^ + --- + /l2"-3)2 = (1 + /i2 + . . . + /l2«-2)2. 

This equality has been verified before in eq. (l24l ). 
This proves facts 1,2,3 and 4, which in turn implies the results of Lemma 3. □ 

In conclusion, application of Lemma 1, Lemma 2 and Lemma 3 gives the proof to Theorem 1. 
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3. Conclusion 

This note gives a proof of a rank recovery problem where the matrix to be recovered is known 
to be Hankel. It is shown that this heuristic gives the correct answer in case the correct (noiseless) 
entries of the upper triangular part of this matrix are given. 
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